1. Field of the Invention
The present invention relates to a stereoscopic image data structure, a stereoscopic image data recording method, a reproducing method, a recording program, and a reproducing program.
2. Related Art
There have been various types of stereoscopic display devices or three-dimensional display devices that can display moving stereoscopic images. In recent years, there has been an increasing demand for stereoscopic display devices of flat panel types that do not require special glasses or the likes. In a flat-panel display device such as a liquid crystal display device or a plasma display device of a direct view type or a projection type, the pixel locations in the display plane are fixed, and a parallax barrier that controls the light rays from the display panel toward the viewer is provided directly in front of the display panel. Thus, a stereoscopic display device can be produced with relative ease.
Through a parallax barrier, light rays are controlled in such a manner that different images can be seen from different angles even when the same location on the parallax barrier is viewed. More specifically, in a case where only right and left parallaxes (horizontal disparities) are given, a slit sheet or a lenticular sheet (a cylindrical lens array) is used. In a case where vertical disparities are also given, a pinhole array or a lens array is used. Structures with parallax barriers are further classified into binocular types, multiview types, super-multiview types (multiview types under super-multiview conditions), and integral photography (IP) types. The principles of those structures are basically the same as the principles of stereoscopic photography invented almost 100 years ago.
Generally, in a structure of the IP type or the multiview type, the viewing distance is limited, and therefore, a display image is created so that a perspectively projected image can be actually viewed at the viewing distance. In the structure of the IP type only with the horizontal disparities (the one-dimensional IP type, see “SID04 Digest 1438” (2004), for example), sets of parallel light rays are formed in a case where the horizontal pitch of the parallax barrier is set at an integral multiple (n) of the horizontal pitch of pixels (this IP type will be hereinafter also referred to as the “parallel-ray one-dimensional IP type”). Accordingly, a parallax component image in which pixel columns forming sets of parallel light rays are integrated is a perspectively projected image with a predetermined viewing distance in the vertical direction while being an orthographically projected image in the horizontal direction. Each parallax component image that is a perspectively projected image in the vertical direction while being an orthographically projected image in the horizontal direction is divided into pixel columns, and the pixel columns are rearranged in an interleaving manner, so as to form a parallax interleaved image (an elemental image array). The parallax interleaved image is displayed on the display plane and is viewed through the parallax barrier. In this manner, a stereoscopic image is obtained through normal projection, which is perspective projection in both the horizontal direction and the vertical direction. This method is described in greater detail in “SID04 Digest 1438” (2004). In a structure of the multiview type, images formed through simple perspective projection are divided into pixel columns and are rearranged in an interleaving manner, so as to form a stereoscopic image with normal projection.
An image-taking device that uses different projecting methods and different projection center distances depending on the direction (the vertical or horizontal direction) is difficult to produce, because a camera or lenses of the same size as each object are required for orthographic projection. Therefore, to obtain orthographic projection data through image taking, a method of converting perspective projection data into orthographic projection data is preferred in practice. As an example of such a method, the “ray space method” that involves interpolation using an “EPI (epipolar plane)” is known.
The parallel-ray one-dimensional IP is more advantageous in viewability than the binocular method. However, in a structure of the parallel-ray one-dimensional IP type, the image format is complicated in terms of projection and division allocation. In a binocular or multiview structure that is one of the simplest stereoscopic display structures, the image format is also simple, and the images from all the viewpoints are formed with the same numbers of vertical and horizontal pixels. Two parallax component images in the case of the binocular type, or nine parallax components images in the case of a nine-lens type, are divided into pixel columns, and the pixel columns are rearranged into a parallax interleaved image to be displayed on the display plane.
Compared with the multiview type with similar resolutions, the number of parallax components images is larger in a structure of the parallel-ray one-dimensional IP type, and the numbers of horizontal pixels (or the horizontal ranges to be used) of the parallax component images vary with the parallax directions. As a result, the image format is complicated. With these facts being taken into consideration, the inventors suggested a method of efficiently recording a stereoscopic image involving a high compression rate with little degradation of image quality (Japanese Patent Application No. 2004-285246).
The cylindrical lenses of a lenticular sheet may extend diagonally, instead of vertically (see JP-A KOKAI No. 2001-501073). The inventors also discovered that the parallel-ray one-dimensional IP type could be applied to a structure of the slanted lens type (Japanese Patent Application No. 2004-32973).
In a case where parallax information is allocated to each sub-pixel in a structure of the multiview type or the parallel-ray one-dimensional IP type, the parallax information is mixed when images in the form of parallax interleaved images are irreversibly compressed by an encoding method such as JPEG or MPEG. As a result, the image quality is degraded at the time of decompression. In a case of reversible (lossless) compression, the problem of image quality degradation is not caused, but the compression rate is much lower than in the case of irreversible (lossy) compression. Also, a method of irreversibly compressing and then decompressing parallax component images independently of one another is easily applied in a structure of a multiview type. However, such a method is not reasonable in a structure of the parallel-ray one-dimensional IP type that involves a large number of parallax component images with different numbers of horizontal pixels. Especially in a case where the lenses extend diagonally with respect to the vertical direction, the data format and processing become more complicated, and it is difficult to achieve a high resolution and a high processing speed at the same time.
As described above, the conventional method of recording a stereoscopic image of a parallel-ray one-dimensional IP type has the problems of image quality degradation with a high compression rate at the time of decompressing.